Workshop in Simulation Debriefing for Educators in Medicine: Creation, implementation, and evaluation of a debriefing curriculum for novice simulation educators.

Objectives: Debriefing is an integral component of simulation education, and effective debriefing education is required to maintain effective simulation programs. However, many educators report financial and logistical barriers to accessing formal debriefing training. Due to limited educator development opportunities, simulation program leaders are often compelled to utilize educators with insufficient debriefing training, which can limit the impact of simulation-based education. To address these concerns, the SAEM Simulation Academy Debriefing Workgroup authored the Workshop in Simulation Debriefing for Educators in Medicine (WiSDEM), a freely available, concise, and ready-to-deploy debriefing curriculum with a target audience of novice educators without formal debriefing training. In this study, we describe the development, initial implementation, and evaluation of the WiSDEM curriculum. Methods: The Debriefing Workgroup iteratively developed the WiSDEM curriculum by expert consensus. The targeted level of content expertise was introductory. The curriculum's educational impact was assessed by surveying participants on their impressions of the curriculum and their confidence and self-efficacy in mastery of the material. Additionally, facilitators of the WiSDEM curriculum were surveyed on its content, usefulness, and future applicability. Results: The WiSDEM curriculum was deployed during the SAEM 2022 Annual Meeting as a didactic presentation. Thirty-nine of 44 participants completed the participant survey, and four of four facilitators completed the facilitator survey. Participant and facilitator feedback on the curriculum content was positive. Additionally, participants agreed that the WiSDEM curriculum improved their confidence and self-efficacy in future debriefing. All surveyed facilitators agreed that they would recommend the curriculum to others. Conclusions: The WiSDEM curriculum was effective at introducing basic debriefing principles to novice educators without formal debriefing training. Facilitators felt that the educational materials would be useful for providing debriefing training at other institutions. Consensus-driven, ready-to-deploy debriefing training materials such as the WiSDEM curriculum can address common barriers to developing basic debriefing proficiency in educators.

Simulation for diversity, equity and inclusion in emergency medicine residency training: A qualitative study.

Background: The last few years have seen an increased focus on diversity, equity, and inclusion (DEI) initiatives across organizations. Simulation has been used in varying degrees for teaching about DEI topics with emergency medicine; however, there are no established best practices or guidelines on this subject. To further examine the use of simulation for DEI teachings, the DEISIM work group was created as a collaboration between the Society of Academic Emergency Medicine (SAEM) Simulation Academy and the Academy for Diversity and Inclusion in Emergency Medicine (ADIEM). This study represents their findings. Method: This qualitative study was conducted using a three-pronged approach. Initial literature search was conducted followed by a call for submission of simulation curricula. These were then followed by five focus groups. Focus groups were recorded, transcribed by a professional transcription service, and then subjected to thematic analysis. Results: Data were analyzed and organized into four broad categories including Learners, Facilitators, Organizational/Leadership, and Technical Issues. Challenges within each of these were identified, as were potential solutions. Select pertinent findings included focused faculty development, a carefully planned approach that utilized DEI content experts and the use of simulation for workplace microaggressions or discriminations. Conclusions: There appears to be a clear role for simulation in DEI teachings. Such curricula, however, should be undertaken with careful planning and input from appropriate and representative parties. More research is needed on optimizing and standardizing simulation-based DEI curricula.

Debriefing Gold: Harnessing the Power of Debriefing Data to Inform Education.

Debriefing is a critical element in healthcare, both in the clinical environment and in the simulation lab. Often, what is said at a debriefing is not recorded, leading to loss of critical data that could be used to inform future simulations, education, and systems improvement. In this perspective piece, we explain the powerful role that capturing debriefing data can have for identifying themes to improve learners' knowledge and skills, as well as inform data-driven systems change and initiatives.

Enhancing Safety of a System-Wide In Situ Simulation Program Using No-Go Considerations.

INTRODUCTION: A large-scale in situ simulation initiative on cardiac arrest in pregnancy was implemented across NYC Health + Hospitals. In situ simulation must be safely balanced with clinical conditions such as through application of no-go considerations or standardized reasons to cancel or postpone the simulation. Our objective is to describe our findings on the application of no-go considerations during this simulation initiative. METHODS: NYC Health + Hospitals/Simulation Center developed an in situ simulation program focused on cardiac arrest in pregnancy, implemented at 11 acute care facilities. The program's toolkit included no-go considerations for in situ simulation safety: situations prompting a need to cancel, reschedule, or postpone a simulation to ensure patient and/or staff safety. RESULTS: Data were collected from June 2018 through December 2019. The simulation sites reviewed the 13 established no-go considerations before each simulation event to assess if the simulation was safe to "go". After the conclusion of the initiative, all data related to no-go considerations were analyzed.Two hundred seventy-four in situ simulations were scheduled and 223 simulations (81%) were completed. Fifty-one no-go events were reported, with 78% identifying a reason by category. Twenty-two percent did not report a reason or category. Four of the 13 suggested no-go considerations were not reported. CONCLUSIONS: The no-go considerations framework promotes standardized and strategic scheduling of in situ simulation. Analysis of no-go consideration application during this system-wide initiative provides a model for the usage of tracking no-go data to enhance safety and inform future simulation planning.

Impact of the PEARLS Healthcare Debriefing cognitive aid on facilitator cognitive load, workload, and debriefing quality: a pilot study.

BACKGROUND: The Promoting Excellence and Reflective Learning in Simulation (PEARLS) Healthcare Debriefing Tool is a cognitive aid designed to deploy debriefing in a structured way. The tool has the potential to increase the facilitator's ability to acquire debriefing skills, by breaking down the complexity of debriefing and thereby improving the quality of a novice facilitator's debrief. In this pilot study, we aimed to evaluate the impact of the tool on facilitators' cognitive load, workload, and debriefing quality. METHODS: Fourteen fellows from the New York City Health + Hospitals Simulation Fellowship, novice to the PEARLS Healthcare Debriefing Tool, were randomized to two groups of 7. The intervention group was equipped with the cognitive aid while the control group did not use the tool. Both groups had undergone an 8-h debriefing course. The two groups performed debriefings of 3 videoed simulated events and rated the cognitive load and workload of their experience using the Paas-Merriënboer scale and the raw National Aeronautics and Space Administration task load index (NASA-TLX), respectively. The debriefing performances were then rated using the Debriefing Assessment for Simulation in Healthcare (DASH) for debriefing quality. Measures of cognitive load were measured as Paas-Merriënboer scale and compared using Wilcoxon rank-sum tests. Measures of workload and debriefing quality were analyzed using mixed-effect linear regression models. RESULTS: Those who used the tool had significantly lower median scores in cognitive load in 2 out of the 3 debriefings (median score with tool vs no tool: scenario A 6 vs 6, p=0.1331; scenario B: 5 vs 6, p=0.043; and scenario C: 5 vs 7, p=0.031). No difference was detected in the tool effectiveness in decreasing composite score of workload demands (mean difference in average NASA-TLX -4.5, 95%CI -16.5 to 7.0, p=0.456) or improving composite scores of debriefing qualities (mean difference in DASH 2.4, 95%CI -3.4 to 8.1, p=0.436). CONCLUSIONS: The PEARLS Healthcare Debriefing Tool may serve as an educational adjunct for debriefing skill acquisition. The use of a debriefing cognitive aid may decrease the cognitive load of debriefing but did not suggest an impact on the workload or quality of debriefing in novice debriefers. Further research is recommended to study the efficacy of the cognitive aid beyond this pilot; however, the design of this research may serve as a model for future exploration of the quality of debriefing.

Procedural task trainer gaps in emergency medicine: A rift in the simulation universe.

Objectives: We identified and quantified the gap between emergency medicine (EM) procedures currently taught using simulation versus those that educators would teach if they had better procedural task trainers. Additionally, we endeavored to describe which procedures were taught using homemade models and the barriers to creation and use of additional homemade models. Methods: Using a modified Delphi process, we developed a survey and distributed it to a convenience sample of EM simulationists via the Society for Academic Emergency Medicine Simulation Academy listserv. Survey items asked participants to identify procedures they thought should be taught using simulation ("most important"), do teach using simulation ("most frequent"), would teach if a simulator or model were available ("most needed"), and do teach using simulation with "homemade" models ("most frequent homemade"). Results: Thirty-seven surveys were completed. The majority of respondents worked at academic medical centers and were involved in simulation-based education for at least 6 years. Three procedures ranked highly in overall teaching importance and currently taught categories. We identified four procedures that ranked highly as both important techniques to teach and would teach via simulation. Two procedures were selected as the most important procedures that the participants do teach via simulation but would like to teach in an improved way. We found 14 procedures that simulationists would teach if an adequate model was available, four of which are of high importance. Conclusions: This study captured data to illuminate the procedural model gap and inform future interventions that may address it and meet the overarching objective to create better and more readily available procedure models for EM simulation educators in the future. It offers an informed way of prioritizing procedures for which additional homemade models should be created and disseminated as well as barriers to be aware of and to work to overcome. Our work has implications for learners, educators, administrators, and industry.

The perceived work environment and well-being: A survey of emergency health care workers during the COVID-19 pandemic.

BACKGROUND: During the COVID-19 pandemic, health care provider well-being was affected by various challenges in the work environment. The purpose of this study was to evaluate the relationship between the perceived work environment and mental well-being of a sample of emergency physicians (EPs), emergency medicine (EM) nurses, and emergency medical services (EMS) providers during the pandemic. METHODS: We surveyed attending EPs, resident EPs, EM nurses, and EMS providers from 10 academic sites across the United States. We used latent class analysis (LCA) to estimate the effect of the perceived work environment on screening positive for depression/anxiety and burnout controlling for respondent characteristics. We tested possible predictors in the multivariate regression models and included the predictors that were significant in the final model. RESULTS: Our final sample included 701 emergency health care workers. Almost 23% of respondents screened positive for depression/anxiety and 39.7% for burnout. Nurses were significantly more likely to screen positive for depression/anxiety (adjusted odds ratio [aOR] 2.04, 95% confidence interval [CI] 1.11-3.86) and burnout (aOR 2.05, 95% CI 1.22-3.49) compared to attendings. The LCA analysis identified four subgroups of our respondents that differed in their responses to the work environment questions. These groups were identified as Work Environment Risk Group 1, an overall good work environment; Risk Group 2, inadequate resources; Risk Group 3, lack of perceived organizational support; and Risk Group 4, an overall poor work environment. Participants in the two groups who perceived their work conditions as most adverse were significantly more likely to screen positive for depression/anxiety (aOR 1.89, 95% CI 1.05-3.42; and aOR 2.04, 95% CI 1.14-3.66) compared to participants working in environments perceived as less adverse. CONCLUSIONS: We found a strong association between a perceived adverse working environment and poor mental health, particularly when organizational support was deemed inadequate. Targeted strategies to promote better perceptions of the workplace are needed.

Hospital-wide cardiac arrest in situ simulation to identify and mitigate latent safety threats.

BACKGROUND: Cardiac arrest resuscitation requires well-executed teamwork to produce optimal outcomes. Frequency of cardiac arrest events differs by hospital location, which presents unique challenges in care due to variations in responding team composition and comfort levels and familiarity with obtaining and utilizing arrest equipment. The objective of this initiative is to utilize unannounced, in situ, cardiac arrest simulations hospital wide to educate, evaluate, and maximize cardiac arrest teams outside the traditional simulation lab by systematically assessing and capturing areas of opportunity for improvement, latent safety threats (LSTs), and key challenges by hospital location. METHODS: Unannounced in situ simulations were performed at a city hospital with multidisciplinary cardiac arrest teams responding to a presumed real cardiac arrest. Participants and facilitators identified LSTs during standardized postsimulation debriefings that were classified into equipment, medication, resource/system, or technical skill categories. A hazard matrix was used by multiplying occurrence frequency of LST in simulation and real clinical events (based on expert opinion) and severity of the LST based on agreement between two evaluators. RESULTS: Seventy-four in situ cardiac arrest simulations were conducted hospital wide. Hundreds of safety threats were identified, analyzed, and categorized yielding 106 unique latent safety threats: 21 in the equipment category, 8 in the medication category, 41 in the resource/system category, and 36 in the technical skill category. The team worked to mitigate all LSTs with priority mitigation to imminent risk level threats, then high risk threats, followed by non-imminent risk LSTs. Four LSTs were deemed imminent, requiring immediate remediation post debriefing. Fifteen LSTs had a hazard ratio greater than 8 which were deemed high risk for remediation. Depending on the category of threat, a combination of mitigating steps including the immediate fixing of an identified problem, leadership escalation, and programmatic intervention recommendations occurred resulting in mitigation of all identified threats. CONCLUSIONS: Hospital-wide in situ cardiac arrest team simulation offers an effective way to both identify and mitigate LSTs. Safety during cardiac arrest care is improved through the use of a system in which LSTs are escalated urgently, mitigated, and conveyed back to participants to provide closed loop debriefing. Lastly, this hospital-wide, multidisciplinary initiative additionally served as an educational needs assessment allowing for informed, iterative education and systems improvement initiatives targeted to areas of LSTs and areas of opportunity.

In their own words: Experiences of emergency health care workers during the COVID-19 pandemic.

BACKGROUND: During the COVID-19 pandemic, a substantial number of emergency health care workers (HCWs) have screened positive for anxiety, depression, risk of posttraumatic stress disorder, and burnout. The purpose of this qualitative study was to describe the impact of COVID-19 on emergency care providers' health and well-being using personal perspectives. We conducted in-depth interviews with emergency physicians, emergency medicine nurses, and emergency medical services providers at 10 collaborating sites across the United States between September 21, 2020, and October 26, 2020. METHODS: We developed a conceptual framework that described the relationship between the work environment and employee health. We used qualitative content analysis to evaluate our interview transcripts classified the domains, themes, and subthemes that emerged from the transcribed interviews. RESULTS: We interviewed 32 emergency HCWs. They described difficult working conditions, such as constrained physical space, inadequate personnel protective equipment, and care protocols that kept changing. Organizational leadership was largely viewed as unprepared, distant, and unsupportive of employees. Providers expressed high moral distress caused by ethically challenging situations, such as the perception of not being able to provide the normal standard of care and emotional support to patients and their families at all times, being responsible for too many sick patients, relying on inexperienced staff to treat infected patients, and caring for patients that put their own health and the health of their families at risk. Moral distress was commonly experienced by emergency HCWs, exacerbated by an unsupportive organizational environment. CONCLUSIONS: Future preparedness efforts should include mechanisms to support frontline HCWs when faced with ethical challenges in addition to an adverse working environment caused by a pandemic such as COVID-19.

Leveling the field: Development of reliable scoring rubrics for quantitative and qualitative medical education research abstracts.

BACKGROUND: Research abstracts are submitted for presentation at scientific conferences; however, criteria for judging abstracts are variable. We sought to develop two rigorous abstract scoring rubrics for education research submissions reporting (1) quantitative data and (2) qualitative data and then to collect validity evidence to support score interpretation. METHODS: We used a modified Delphi method to achieve expert consensus for scoring rubric items to optimize content validity. Eight education research experts participated in two separate modified Delphi processes, one to generate quantitative research items and one for qualitative. Modifications were made between rounds based on item scores and expert feedback. Homogeneity of ratings in the Delphi process was calculated using Cronbach's alpha, with increasing homogeneity considered an indication of consensus. Rubrics were piloted by scoring abstracts from 22 quantitative publications from AEM Education and Training "Critical Appraisal of Emergency Medicine Education Research" (11 highlighted for excellent methodology and 11 that were not) and 10 qualitative publications (five highlighted for excellent methodology and five that were not). Intraclass correlation coefficient (ICC) estimates of reliability were calculated. RESULTS: Each rubric required three rounds of a modified Delphi process. The resulting quantitative rubric contained nine items: quality of objectives, appropriateness of methods, outcomes, data analysis, generalizability, importance to medical education, innovation, quality of writing, and strength of conclusions (Cronbach's α for the third round = 0.922, ICC for total scores during piloting = 0.893). The resulting qualitative rubric contained seven items: quality of study aims, general methods, data collection, sampling, data analysis, writing quality, and strength of conclusions (Cronbach's α for the third round = 0.913, ICC for the total scores during piloting = 0.788). CONCLUSION: We developed scoring rubrics to assess quality in quantitative and qualitative medical education research abstracts to aid in selection for presentation at scientific meetings. Our tools demonstrated high reliability.

Stroke Code From EMS to Thrombectomy: An Interdisciplinary In Situ Simulation for Prompt Management of Acute Ischemic Stroke.

INTRODUCTION: Treatment of acute ischemic stroke is challenging because it requires prompt management, interdisciplinary collaboration, and adherence to specific guidelines. This resource addresses these challenges by providing in situ simulated practice with stroke codes by practicing clinicians at unannounced times. METHODS: An emergency department team was presented with a 55-year-old simulated patient with speech difficulty and right-sided weakness. The team had to assess her efficiently and appropriately, including activating the stroke team via the hospital paging system. The stroke team responded to collaboratively coordinate evaluation, obtain appropriate imaging, administer thrombolytic therapy, and recognize the need for thrombectomy. Learners moved through the actual steps in the real clinical environment, using real hospital equipment. Upon simulation completion, debriefing was utilized to review the case and team performance. Latent safety threats were recorded, if present. Participants completed an evaluation to gauge the simulation's effectiveness. RESULTS: Six simulations involving 40 total participants were conducted and debriefed across New York City Health + Hospitals. One hundred percent of teams correctly identified the presenting condition and assessed eligibility for thrombolytic and endovascular therapy. Evaluations indicated that 100% of learners found the simulation to be an effective clinical, teamwork, and communication teaching tool. Debriefing captured several latent safety threats, which were rectified by collaboration with hospital leadership. DISCUSSION: Impromptu, in situ simulation helps develop interdisciplinary teamwork and clinical knowledge and is useful for reviewing crucial times and processes required for best-practice patient care. It is particularly useful when timely management is essential, as with acute ischemic stroke.

Helping Mothers Survive: Program Evaluation and Early Outcomes of Maternal Care Training in the Dominican Republic.

Introduction: In 2017, approximately 295,000 women died during and immediately following pregnancy and childbirth worldwide, with 94% of these deaths occurring in low-resource settings. The Dominican Republic (DR) exhibits one of the highest maternal mortality ratios in the region of Latin America and the Caribbean despite the fact that 99% of registered births in the country are reportedly attended by a skilled birth attendant. This paradox implies that programs to support healthcare worker knowledge and skills improvement are vital to improving maternal health outcomes in the DR. Helping Mothers Survive (HMS) is a provider training program developed by Jhpiego and global partners. The goal of HMS is to combat maternal mortality by contributing to quality improvement efforts that reinforce maternal health skills of local healthcare workers. Methods: An international, multisectoral group of stakeholders collaborated in the implementation of two HMS curricula, Bleeding After Birth (BAB) and pre-eclampsia & eclampsia (PE&E). Demographic information as well as pre- and post-training knowledge scores were recorded for each participant. Knowledge score improvement was assessed in order to support effectiveness of the program on knowledge acquisition of healthcare workers. Results: Three hundred and twenty healthcare workers participated in the HMS training workshops between October 2016-August 2020. Of the 320 participants, 132 were trained as master trainers. The majority of participants identified as attending physicians, followed by residents/interns, nurses, students, and "other." A significant improvement in knowledge scores was observed for both the BAB and PE&E curricula, with a 21.24 and 30.25% change in average score (pre- to post-test), respectively. In response to COVID-19 pandemic restrictions, flexibility of the local team led to a PE&E virtual training pilot workshop in August 2020. Discussion/Conclusions: Simulation-based training improved the knowledge levels of healthcare workers for both HMS curricula. These results suggest that simulation-based workshops have an impact on knowledge acquisition and skills of healthcare workers immediately following training. For the PE&E curriculum, no significant difference in knowledge acquisition was observed between in-person and virtual training sessions. The ongoing pandemic poses challenges to program implementation; however, these preliminary results provide evidence that conducting virtual workshops may be a viable alternative to in-person training.

Debrief it all: a tool for inclusion of Safety-II.

Safety science in healthcare has historically focused primarily on reducing risk and minimizing harm by learning everything possible from when things go wrong (Safety-I). Safety-II encourages the study of all events, including the routine and mundane, not only bad outcomes. While debriefing and learning from positive events is not uncommon or new to simulation, many common debriefing strategies are more focused on Safety-I. The lack of inclusion of Safety-II misses out on the powerful analysis of everyday work.A debriefing tool highlighting Safety-II concepts was developed through expert consensus and piloting and is offered as a guide to encourage and facilitate inclusion of Safety-II analysis into debriefings. It allows for debriefing expansion from the focus on error analysis and "what went wrong" or "could have gone better" to now also capture valuable discussion of high yield Safety-II concepts such as capacities, adjustments, variation, and adaptation for successful operations in a complex system. Additionally, debriefing inclusive of Safety-II fosters increased debriefing overall by encouraging debriefing when "things go right", not historically what is most commonly debriefed.

Virtual Mentoring: Two Adaptive Models for Supporting Early-career Simulation Investigators in the Era of Social Distancing.

BACKGROUND: Early-career simulation investigators identify limited mentorship as a common barrier to disseminating scholarship and launching a successful academic career in emergency medicine (EM). Conferences often bridge this gap, but the COVID-19 pandemic has forced their indefinite delay. Virtual solutions are needed to capitalize on the breadth of national simulation research experts and grow mentorship in a postpandemic world. METHODS: We developed two complementary innovations to facilitate scholarship development and minimize COVID-associated career challenges resulting from social distancing requirements. The e-fellows forum (FF) provides a capstone experience for works-in-progress and the e-consultation service (CS) supports simulation research during the earlier project stages of design and development. In conjunction with the Society for Academic Medicine's Simulation Academy, we applied videoconferencing technology for both of these novel, virtual innovations. We analyzed corresponding chat transcripts and detailed field notes for emerging themes. In addition, we collected quantitative data via participant surveys regarding their experiences and impact on their projects. RESULTS: Nine simulation fellows presented at the FF and seven junior simulation investigators participated in the CS sessions. Most preferred the virtual format (56% FF, 66% CS) and found the sessions to be helpful in project advancement (66% FF, 100% CS). COVID-19 affected most projects (89% FF, 67% CS). We identified three themes via qualitative analysis: design concerns and inquiries, validation or support shown by mentors and peers, and professional cohesion. CONCLUSIONS: Participants felt that both virtual mentorship innovations advanced their simulation research projects and fostered a sense of professional cohesion within a greater community of practice. These benefits can be powerful at a time where simulation researchers in EM feel disconnected in an era of social distancing. Our future work will include adaptations to a hybrid model with both virtual and in-person modalities as well as creation of more e-mentorship opportunities, thus broadening the early-career simulation research community of practice.